Thanks for the submission. could you please run.sh (perhaps add it as comments in the PR instead) and add a CI test for your algorithm and sparse-small dataset in .github/neurips23.yaml?

Hi @harsha-simhadri, from what I understood I removed the run.sh and added to the workflow, but I'm not sure how to test if the adding to the workflow worked.

I see the CI test for NLE is green. Thanks for that.
I am running NLE on sparse-full on the competition spec VM. Will merge once that completes.

These are the results I see on the competition spec VM

nle,pisa,sparse-full,10,2993.964456425476,0.0,1000000.0,10371632.0,3464.1800699206683,0,0,sparse,0.8624498567335243
nle,pisa,sparse-full,10,3001.2004210420996,0.0,1000000.0,10371632.0,3455.827850510125,0,0,sparse,0.8634527220630372
nle,pisa,sparse-full,10,2836.924317267769,0.0,1000000.0,10371632.0,3655.942436275099,0,0,sparse,0.8725787965616046
nle,pisa,sparse-full,10,2567.266042978288,0.0,1000000.0,10371632.0,4039.95216170423,0,0,sparse,0.8923065902578797
nle,pisa,sparse-full,10,2455.7393810658486,0.0,1000000.0,10371632.0,4223.425368329789,0,0,sparse,0.8980085959885387
nle,pisa,sparse-full,10,2365.8710581835744,0.0,1000000.0,10371632.0,4383.853449715448,0,0,sparse,0.902349570200573
nle,pisa,sparse-full,10,2275.55007253081,0.0,1000000.0,10371632.0,4557.8570760541115,0,0,sparse,0.9059455587392551
nle,pisa,sparse-full,10,2208.434808525321,0.0,1000000.0,10371632.0,4696.372272326953,0,0,sparse,0.9083810888252148
nle,pisa,sparse-full,10,2087.793570011214,0.0,1000000.0,10371632.0,4967.7478410589665,0,0,sparse,0.9110171919770774
nle,pisa,sparse-full,10,1700.6021543377224,0.0,1000000.0,10371632.0,6098.799753690244,0,0,sparse,0.9195845272206304

Does this agree with your observations? IF so, I can merge this PR

Hi @harsha-simhadri , the numbers are smaller than what I had expected and I found a bug on the number of threads (it was hardcoded to 4 threads instead of CPU count), could you please run the new version?

This is what I see with the latest commit

nle,pisa,sparse-full,10,3024.789817796044,0.0,1000000.0,4133388.0,1366.5042032612087,0,0,sparse,0.8624498567335243
nle,pisa,sparse-full,10,3006.411412028175,0.0,1000000.0,4133388.0,1374.8577401825214,0,0,sparse,0.8634527220630372
nle,pisa,sparse-full,10,2849.467293396349,0.0,1000000.0,4133388.0,1450.5827140318972,0,0,sparse,0.8725787965616046
nle,pisa,sparse-full,10,2567.785514672901,0.0,1000000.0,4133388.0,1609.70921300899,0,0,sparse,0.8923065902578797
nle,pisa,sparse-full,10,2459.149560937577,0.0,1000000.0,4133388.0,1680.820095555352,0,0,sparse,0.8980085959885387
nle,pisa,sparse-full,10,2367.564881465038,0.0,1000000.0,4133388.0,1745.8393779866674,0,0,sparse,0.902349570200573
nle,pisa,sparse-full,10,2284.1869903221677,0.0,1000000.0,4133388.0,1809.566387302213,0,0,sparse,0.9059455587392551
nle,pisa,sparse-full,10,2207.291414343687,0.0,1000000.0,4133388.0,1872.6063867869555,0,0,sparse,0.9083810888252148
nle,pisa,sparse-full,10,2091.175372849145,0.0,1000000.0,4133388.0,1976.5860164891,0,0,sparse,0.9110171919770774
nle,pisa,sparse-full,10,1701.0052918629858,0.0,1000000.0,4133388.0,2429.9677489380438,0,0,sparse,0.9195845272206304

Hi @harsha-simhadri, sorry for this, the result seem weird because there was no difference between 4 hard-coded threads and what was supposed to be 8. I've hardcoded it to 8 now and if there's no difference I probably had hardcoded somewhere to 8 before and I'm not finding (code is not as clean as it could be as you're seeing). Can you please try with the newest commit and if there's no difference I think I will just leave as is.

@cadurosar Let me merge this to make way for other submissions. Feel free to investigate further and submit any config changes before Oct 31 deadline.